Lecture 12 — November 3 Lecturer : Lester Mackey Scribe

In the first part of the course, we focused on optimal inference in the setting of point estimation (see Figure 12.2). We formulated this problem in the framework of decision theory and focused on finite sample criteria of optimality. We immediately discovered that uniform optimality was seldom attainable in practice, and thus, we developed our theory of optimality along two lines: constraining and collapsing. To restrict ourselves to interesting subclasses of estimators, we first introduced the notion of unbiasedness, which lead us to UMRUEs/UMVUEs. Then we considered certain symmetry constraints in the context of location invariant decision problems—this was formalized via the concept of equivariance, which led us to MREs. The situation is similar in hypothesis testing: to develop a useful notion of optimality, we will need to impose constraints on tests. These constraints arise in the form of risk bounds, unbiasedness, and equivariance. Another way to achieve optimality was to collapse the risk function. We introduced the notions of average risk (optimized by Bayes estimators) and worst case risk (optimized by minimax estimators). We saw that Bayes estimators have many desirable properties and provide tools to reason about both concepts. We also considered a few different model families such as exponential family and location and scale family. We came out with certain notions of optimality, optimal unbiased estimator was easy to find for exponential family with complete sufficient statistics and for locationscale family, we defined the best equivariant estimators.